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Introduction 

Much of the national attention on science, technology, engineering, and mathematics (STEM) 
education tends to concentrate on science and mathematics, with its emphasis on standardized 
test scores. However as the National Academy of Engineering Committee on K-12 Engineering 
Education stressed, engineering can contribute to the development of an effective and 
interconnected STEM education system (Katehi, Pearson, & Feder, 2009). In addition, 
engineering can provide authentic learning contexts for science, technology, and mathematics. 
Numerous K-12 engineering initiatives have emerged across the U.S. developing curriculum and 
conducting teacher professional development (Brophy, Klein, Portsmore, & Rogers, 2008). The 
focus of pre-college engineering education has largely been on process, with engineering content 
or concepts playing at best a secondary role. 

The Standards for Technological Literacy (STL) (2000), for example, has been cited by many as 
providing direction for pre-college engineering, with its design-oriented standards. However, the 
STL do not specify engineering content and focuses only on the design process. In addition, 
numerous studies have been conducted to identify engineering-oriented outcomes and 
competencies (Childress & Rhodes, 2008; Dearing & Daugherty, 2004; Harris & Rogers, 2008). 
However, these studies have resulted in lists that focus heavily on process and the interpersonal 
skills associated with engineering (communication, teamwork, etc.). For example, Childress and 
Sanders (2007) examined the related literature and engineering curricular materials, concluding 
that it is “challenging to create a framework that might be helpful in developing ‘engineering’ 
instructional materials for secondary schools” (p. 5). 

Within teacher professional development this lack of focus on engineering concepts is also 
evident. One key problem confronting engineering professional development observed by 
Daugherty (2009) in a multiple case study project is the lack of a well-defined concept base. This 
case study, funded by the National Center for Engineering and Technology Education (NCETE), 
included the following professional development projects: Engineering the Future: Science, 
Technology, and the Design Process™, Project Lead the Way™, Mathematics Across the Middle 
School MST Curriculum, The Infinity Project m , and INSPIRES. The findings from the individual 
case studies were compared and summarized across the study’s research questions, which 
included a focus on identifying the fundamental content knowledge. As evidenced by the case 
studies, engineering content is not well-defined for secondary level education. The projects’ 
leaders, instructors, and participating teachers discussed engineering in terms of the design 
process and were unable to articulate the associated content or concepts. 

This void poses serious problems for curriculum and professional development, as well as for 
research. In the absence of a conceptual base, materials tend to focus solely on design-based 
activities, lacking a focus on conceptual learning. In order to address this gap Custer, Daugherty, 
and Meyer (in press), in a study funded by NCETE, identified thirteen engineering concepts 
deemed to be core to engineering and appropriate for the secondary level. To further this effort 
beyond a list of thirteen concepts, the current study, also funded by NCETE, convened a focus 
group comprised of pre-college engineering education, cognitive science, and/or concept 
mapping experts to discuss how and in what ways concept mapping could be used in secondary 
level engineering education. The study’s research questions were; 
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1. How can concept mapping be used to facilitate learning in secondary level engineering 
education? 

2. How can concept mapping be used to assess learning in secondary level engineering 
education? 


Background 

Using an emergent qualitative research design, the Custer, Daugherty, and Meyer (in press) study 
provided an in-depth analysis of a broad range of engineering-related literature and focus groups 
with engineering educators and engineers. Four types of documents were reviewed including: (a) 
engineering and technology philosophy writings, (b) curriculum materials focused on secondary 
level engineering, (c) curriculum standards documents developed for the STEM disciplines and 
National Academy of Engineering reports, and (d) survey research studies relevant to K-12 
engineering. Two of the three researchers, alternating the pair of researchers, independently 
reviewed each set of documents and identified “engineering themes.” 

Engineering themes were those elements in the narrative that were described as important to 
engineering and applicable across various engineering disciplines, as informed by the philosophy 
of engineering and technology literature. The decision was made to be inclusive, retaining 
themes that would later be analyzed and refined through a systematic, analytical procedure 
employed by the research team. The process generated over 100 themes. Each of the engineering 
themes identified in the literature were subjected to three criteria, including whether they were 
(a) appropriate for engineering at the secondary level, (b) core to engineering, and (c) 
conceptually robust. All three researchers independently applied the criteria to the list of themes. 
In order to be included in the listing of core engineering concepts, the theme was required to 
meet all three criteria by all three researchers on a consensus basis. 

A series of focus groups was also conducted with 21 engineering educators and engineers. 

During these sessions, the participants generated their own lists of engineering concepts using 
the same criteria that were used by the researchers. They were then invited to reflect on 
similarities and differences between the lists that they generated and the outcomes of the 
literature -based analysis. This was another important input to the research process that resulted in 
the set of thirteen concepts that are outlined in Table 1. In addition to the list of concepts, column 
two contains a set of descriptive terms associated with each concept. These terms were drawn 
directly from the document sources and were used to define, clarify, or illustrate the concepts. 
The remaining columns provide an indication of where the concept was located within the five 
inputs. Careful records were maintained to track the sources of themes and concepts derived 
from all five inputs throughout the analysis. 
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Table 1 

Core Engineering Concepts and Presence in Data Sources 


Concept 

Terms 

Curriculum 

Philosophy 

Standards 

Focus 

Groups 

Survey 

Studies 

analysis 

risk, cost/benefit, life-cycle, failure, mathematical, decision, 
economic 

P 

P 

P 

P 

P 

constraints 

criteria, specifications, limitations, requirements 

P 

P 

P 

P 

P 

design 

iterative, technological, analysis based, experimental, ergonomic, 
universal 

P 

P 

P 

P 

P 

efficiency 

key engineering goal, guiding principle 

P 

P 

P 

NP 

NP 

experimentation 

testing, test development, trial and error 

P 

P 

P 

P 

P 

functionality 

key engineering goal, usefulness, practicality 

P 

P 

NP 

P 

P 

innovation 

creativity, improvement, refinement, invention 

P 

P 

P 

P 

P 

modeling 

mathematical, computer-based, technical drawing, physical 

P 

P 

P 

P 

P 

optimization 

improvement, refinement, balancing, decision heuristics 

P 

P 

P 

P 

P 

prototyping 

physical and process modeling and evaluation, preliminary 

P 

P 

P 

P 

P 

systems 

input/output, process, feedback, component design and interaction, 
subsystems 

P 

P 

P 

P 

P 

trade-offs 

conflicting constraints, negotiation, competing requirements or 
criteria 

P 

P 

P 

P 

P 

visualization 

imagery, spatial and abstract representation, sketching 

P 

P 

P 

NP 

P 


Note. P indicates concept present in data source, NP indicates concept absent from data source 


In addition to this list of concepts, the researchers identified issues associated with developing an 
engineering concept base. They determined that a logical next step to further define and use the 
concepts was to develop concept maps, which provide a picture of the way learners structure 
their knowledge of engineering. Concept maps have been well documented as research and 
evaluation tools, particularly in science education (Markham & Mintzes, 1994; Safayeni, 
Derbentseva, & Canas, 2005), guiding assessment (McClure, Sonak, & Suen, 1999) and 
instructional materials (Cliburn, 1986) and as an aid to help students “learn how to learn” 
(Novak, 1990). The current study was designed to investigate the use of concept mapping in 
secondary level engineering education. The primary goal of the focus group was to explore the 
use of concept mapping to facilitate and assess learning in engineering education. 
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Review of Literature 


Concepts are primarily used to categorize and to communicate. Categorization is the process by 
which mental representations (concepts) determine whether or not some entity is a member of a 
category. Categorization enables a large variety of subordinate functions, such as using relevant 
knowledge to understand a situation and predict an outcome (Medin & Rips, 2005). Concepts are 
organizing ideas that are timeless, universal, abstract and broad, represented by one or two 
words, and examples of which share common attributes (Erickson, 2002; Tennyson & 
Cocchiarella, 1986). According to Medin, Lynch, and Solomon (2000), there is recognition that 
there are different concepts and that it is possible to make useful distinctions among them based 
on structure, process, and content. Building conceptual knowledge is essential for learning as it 
requires understanding the operational structure of something and how it relates to associated 
concepts. Conceptual knowledge can be “thought of as a connected web of knowledge, a 
network in which the linking relationships are as prominent as the discrete pieces of information” 
(Hiebert & Lefevre, 1986, p. 3-4). 


Concept Maps 


Concept maps demonstrate how people visualize relationships between various concepts 
(Plotnick, 1997; Wheeldon & Faubert, 2009). In its traditional form, a concept map is a graphical 
node-arc representation of concepts and their relationships with each other. The nodes of the map 
contain the concepts. The links between the nodes captures the relationships among the concepts. 
Labeling the links provides information about the nature of the relationships (Turns, Atman, & 
Adams, 2000). The links between the concepts can be one-way, two-way, or non-directional. The 
concepts and the links may be categorized, and the concept map may show temporal or causal 
relationships between concepts (Plotnick, 1997). For example, Figure 1 is an engineering 
concept map that indicates the relationships between concepts with words linking the nodes. 



invoh'es includes creates 



Figure 1. Concept map for engineering (Turns, Atman & Adams, 2000) 
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Concept mapping has its roots in cognitive psychology (Wheeldon & Faubert, 2009). However, 
different theoretical bases for concept maps have resulted in different terms such as semantic 
networks, knowledge maps, and mind maps (Turns, Atman, & Adams, 1999; Wheeldon & 
faubert, 2009; Wycoff, 1991). According to semantic theory, knowledge is stored in a network 
format where concepts are linked to each other. This is referred to as the associative network of 
knowledge and the more interconnected the knowledge representation, the higher the probability 
that a person will recall information when required. From a constructivist’s learning theory 
perspective, the learner attains new knowledge by integrating new information with existing 
knowledge structures. Therefore, the network mapping of concepts and their relationships 
externalizes how knowledge is integrated mentally. 

According to Ruiz-Primo and Shavelson (1996), Ausubel’s (1968) theory provides guidance as 
to what constitutes a legitimate concept map. They argued that the maps should be hierarchical 
with super-ordinate concepts at the apex; labeled with appropriate linking words; and cross 
linked so that relationships between sub-branches of the hierarchy are identified. Novak and 
Gowin (1984) further articulated that the hierarchical structure of a concept map should develop 
as new concepts are subsumed by more general inclusive concepts. The expansion of the 
hierarchy is governed by the principles of progressive differentiation, so that new concepts and 
links are added to the hierarchy, either by creating new branches, or by differentiating existing 
ones further. 


Wheeldon and Faubert (2009) argued, however, that in order for concept maps to achieve their 
full capability of translating knowledge there must be flexibility in the traditional form of 
concept maps. These traditional forms tend to constrain concepts maps to include clear and 
unique concepts, lines suggesting hierarchical relationships, and linking words. They presented 
an example represented in Figure 2 of a free form concept map indicating where values come 
from. They reasoned that although this map does not contain a clear hierarchy, linking words, or 
directional arrows, it does offer a view of an individual’s understanding. It might not be 
appropriate to attempt to use this concept map alone to understand how an individual perceives 
the origin of his or her values, but this map might give way to more qualitative coding schemes 
or assist in the development of subsequent data collection approaches, including interviews and 
focus groups. 


Religion 


Values 


Basketball 

Coach 


Figure 2. Free form concept map (Wheeldon & Faubert, 2009) 

Definitional flexibility broadens the use of concept maps so that it plays a larger role in 
knowledge translation and thus increasing its use to qualitative researchers as a tool to organize 
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research, reduce data, analyze themes, and present findings (Daley, 2004; Ebener et al., 2006). 
Thus concept maps have evolved into other types of graphical representations such as spider 
maps and chain maps. 

Concept Map Research 

A substantial amount of research has been done both in education on concept maps (Safayani, 
Derbentseva, Canas, 2005). For example, Willerman and MacHarg (1991) examined the use of 
concept maps as an advance organizer for eighth-grade students in a science unit dealing with 
physical and chemical properties. They found that using concept maps at the beginning of a unit 
resulted in a significant increase on a test administered when the units were completed. A study 
conducted by Kinchin (2000) highlighted the positive effect on students who used concept maps 
finding that when concept maps are produced by students they are more beneficial and have the 
potential to reveal misconceptions in learning that are not captured on traditional assessment 
tools. The conceptual understanding by college students in chemistry laboratories was assessed 
by Markow and Lonning (1998) using concept maps. Among their findings was that students 
have a stronger positive attitude when using concept maps, as well as a better understanding of 
chemistry laboratory concepts. Concepts maps also have been shown to be more effective in 
promoting knowledge retention than attending classes (Poole & Davis, 2006). 

Studies have also compared concepts maps constructed by novices and those constructed by 
experts (i.e., Markham & Mintzes, 1994; Williams, 1998). These studies reveal significant 
differences in the concept maps constructed by experts and those constructed by novices. 
Lederman, Gess-Newsome, and Latz (1994) assessed the development and changes in twelve 
preservice teachers’ subject matter and pedagogical knowledge structures as they proceeded 
through a professional teacher education program. They found that preservice teachers’ initial 
knowledge structure representations were typically linear and lacked coherence. The preservice 
teachers’ knowledge structures however were susceptible to change as a consequence of the act 
of teaching. In addition, the subject matter and pedagogical knowledge structures were reported 
to exert separate influence on classroom practice, with the pedagogy knowledge structure having 
primary influence on instructional decisions. The complexity of the subject matter knowledge 
structure was more critical in determining whether the structure directly influences classroom 
practice. 

Concept Mapping in Assessment 

Concept maps have been explored as teaching and assessment tools. Concept maps have be used 
as a procedure to measure student’s declarative knowledge; however, with certain limitations. 
According to Ruiz-Primo and Shavelson (1996), concepts maps are more directly related to the 
knowledge of facts and concepts and how the concept in a domain are related, than how they are 
used in problem solving. Therefore, they are unable to indicate what students are able to do with 
that knowledge in a certain domain. Turns, Atman, and Adams (2000) stipulated that using 
concept maps as a part of an assessment approach requires consideration of how the maps will be 
constructed and how they will be interpreted. 

Turns, Atman, and Adams (2000) described how concept maps were used as a classroom 
assessment tool to “explore the incoming conceptions and to verify progress in the development 



of both technically sophisticated vocabulary and interconnections among the terms” (p.166). 

The concept maps generated in the middle of the course as compared to the concept maps 
generated at the beginning of the course included more concepts and more cross-links. The maps 
also contained much more sophisticated domain-relevant terminology distributed across an 
increased number of levels of detail. These differences in the maps suggested that the students 
were starting to see interconnections and were making progress in developing an integrated 
understanding of, in this case, human factors. 

For classes with novel formats and ill-defined learning objectives, such as product dissection 
courses and undergraduate research experiences, concept maps can be used as an exploratory 
assessment of what students’ perceive they are learning in the course, apart from the stated 
expectations. Turns, Atman, and Adams (2000) indicated that when concept maps are used for 
this purpose it provides an open-ended means for students to communicate what they see as 
important concepts, and it can also convey their perceptions of the relationships among these 
concepts. The outcome of this analysis can result in the refining of the learning objectives and 
instructional strategies, and also the identification of appropriate assessment tools for 
understanding how the learning objectives are being realized by the students. 

This literature review focused on concept mapping and its applications in research and 
assessment indicates that there is a great potential for the use of concept mapping within pre- 
college engineering education. Science education has pursued the use of concept maps with its 
American Association for the Advancement of Science’s Project 2061 publication, Atlas of 
Science Literacy, Volumes 1 and 2. The Atlas provides concept maps based on the Benchmarks 
for Science Literacy to help guide K-12 science education. To further a similar approach in pre- 
college engineering education it is important to investigate some of the issues involved with a 
group of experts immersed in the research and teaching of pre-college engineering. 

Method 

This qualitative study pursued an expert focus group design. The focus group participants were 
identified by the researcher as possessing expertise in pre-college engineering education, 
cognitive science, and/or concept mapping. In order to maintain a manageable size to elicit 
discussion and participation, eight focus group participants were invited to participate in a six 
hour meeting to discuss the use of concept mapping in pre-college engineering education. An 
email was sent to each of the participants asking for their participation in the focus group and to 
schedule a date for the meeting. In addition to the eight focus group members, two additional 
researchers (Drs. Custer and Dixon) assisted in the facilitation of the meeting and the 
documentation of field notes of the meeting’s proceedings. This enabled sufficient data 
collection coverage with a facilitator and two note-takers throughout each of the major phases of 
the focus group meeting. 

After human subjects consent was obtained from the participants, the focus group meeting 
proceeded in the following primary phases: 

• Discussion of engineering concepts from the Custer, Daugherty, and Meyer study (2009) 
and the Rossouw, Hacker, and de Vries study (2010). 

• Creation of concept maps in pairs using a selected concept from the studies. 



• Discussion of concept mapping as a learning tool and as an assessment tool. 

• Presentation and discussion of the research related to concept mapping focused on how 
concept maps are developed and used in research and what techniques have been 
deployed to use them as assessment tools. 

The focus group meeting was structured around a discussion of engineering concepts appropriate 
for the secondary level. In pairs, the participants selected an engineering concept to map; these 
included design, innovation, analysis, and systems. Each pair presented their maps and included 
a description of their process. Based on this experience and their own expertise, the focus group 
was asked to reflect on the issues related to developing engineering concept maps for learning 
and assessment purposes. The researchers’ field notes, concept maps, and participant’s 
handwritten notes were collected and analyzed according to the study’s two research questions. 

In addition, a short questionnaire was distributed and collected so as to better characterize the 
group according to highest degree obtained, disciplinary area, length of time working and area of 
focus in pre-college engineering education, and concept mapping experience. 

Findings 

Based on the questionnaire, the focus group’s characteristics are described first, and then based 
on an analysis of the researchers’ field notes and the participants’ concept maps and notes, the 
focus group process is described. This is followed by a discussion of the primary themes that 
emerged in reference to the study’s two research questions. 

Focus Group Characteristics 

The focus group averaged 1 1 years of work experience in secondary level engineering education, 
with areas of focus ranging from cognition to curriculum development to teaching. As indicated 
on Table 2, all but two of the participants had engaged in some type of concept mapping before 
and brought these experiences to bear during the meeting. Most of the participants had 
experience with cognitive science. For example, one of the participants was involved with the 
National Research Council’s project, How People Learn (2000). 
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Table 2 

Focus Group Characteristics 


Degree 

Discipline Area 

Years in EE 

Engineering Education Focus Area 

Concept Mapping Experience 

Ph.D. 

Industrial 

Education/Human 

Resource 

Development 

7 

Cognition and Problem Solving 

Yes 

Ph.D. 

Science Education 

30 

Curriculum & Competition 
Programs 

Yes 

Ph.D. 

Chemical 

Engineering 

7 

Curriculum Development and 
Professional Development in 
Technology Education 

No 

Ph.D. 

Learning Sciences 

15 

Electrical Circuits 

Yes 

Ph.D. 

Curriculum Studies 

12 

Curriculum 

No 

M.S. 

Technology 

Education/ 

Mechanical 

Engineering 

10 

Secondary teaching and Teacher 
Preparation 

Yes 

M.S. & 
M.S. 

Environmental 
Science and 
Educational 
Statistics & 
Measurement 

3 

Curriculum Development, 
Professional Development and 
Assessment 

Yes 

M.S. 

Technology in 
Education 

7 

Technological Literacy 

Yes 


Focus Group Process 

After the two concept studies were presented (Custer, Daugherty, & Meyer study, 2009 and 
Rossouw, Hacker, & de Vries study, 2010), the eight participants paired up and selected an 
engineering concept to map; these included design, innovation, analysis, and systems. The 
partners worked together to develop a visual map of their concept on large pieces of paper with 
markers for an hour. Little instruction was provided beyond a general definition of concept maps, 
which was described as a diagram showing the relationships among concepts. Suggestions were 
also provided to help the pairs create their maps, which included: 

• Using a top down approach, 

• Working from general to specific, 

• Using a free association approach by brainstorming nodes and then developing links and 
relationships, 

• Using different colors and shapes for nodes and links to identify different types of 
information, 

• Using different colored nodes to identify prior and new information. 

Each pair presented their maps and included a description of their approach to developing their 
maps. The concept maps developed by the pairs were largely different from each other, as 
indicated in Table 3. These differences appeared to be due to their approach to concept mapping, 
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as well as the concept they selected to map. For example, one pair, who selected innovation as 
their concept to map, had to first reconcile their different approaches to concept mapping. One of 
the partners focused on the process represented by the concept and the other partner focused on 
the relationships connected to the central concept. The pair decided to include both 
representations on their map. Innovation was included on the top of the map in large letters with 
a node connected at the same level labeled invention. Words that defined innovation and 
invention were included as well. Upon reflection, the pair shared their struggle to develop a 
detailed concept map of innovation given its smaller “grain size,” in comparison to other 
concepts such engineering design or systems. 

Table 3 

Participants ’ Approaches 


Concept 

Innovation 

Design 

System 

Analysis 


Participant’s Approach to Concept Mapping 

Included process dimensions and relationships to other concepts. 

Detailed the design process model and questions that a designer must address when 
engaged in the design process. 

Approached mapping as a brainstorming activity spending time individually developing 
ideas and then agreeing on how to proceed with the map 

Approached mapping as a brainstorming activity searching for key terms associated with 
analysis and categorized them as pertaining to what and why questions and process. 


The pairs that developed concept maps for design and systems generated much more intricate 
and detailed maps. For example, the pair that developed the concept map for design created a 
very detailed map that included elements of a design process model and questions that a designer 
must address when engaged in the design process. Both pairs emphasized the need to present and 
describe their maps because, in the absence of interpretation, the maps would be difficult, if not 
impossible, for others to understand. The pair that mapped systems approached it as a 
brainstorming activity, where they spent time individually developing thoughts and ideas and 
then agreed on how to proceed in including these ideas on the piece of paper graphically. 
Similarly, the pair that developed the concept map focused on analysis approached it as a 
brainstorming activity searching for key terms associated with analysis and categorized them as 
pertaining to what and why questions and process. 

Facilitate Learning? 

The focus group participants stressed that concept mapping can be a valuable tool in facilitating 
the learning of engineering concepts in K-12 contexts. They agreed that concept mapping can be 
a useful mechanism for learners to demonstrate their mental models of important engineering 
concepts. Maps can trace levels of sophistication in understanding, they can demonstrate 
procedural or “how to” knowledge, or demonstrate the relationships between ideas. Thus the 
goal of using concept maps as a learning tool needs to be made explicit to the learner. In 
addition, several other important issues emerged during the discussion including: (a) context, (b) 
learning progressions, (c) reflection, (d) misconceptions, and (e) implementation. 
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The focus group discussed at length the importance of context. Mapping engineering concepts in 
isolation or abstraction from a particular context is challenging and perhaps not as helpful to the 
learner. A context, whether it is an engineering design project or a scenario, provides something 
for the learner to reference in building a concept map. Also, the nature of the context helps to 
determine which concepts are important to include in a map. The linking words and descriptors 
included on such a map provide the detail needed to indicate understanding. This indicates that a 
given concept map will be a function of a specific context or project. For example, a concept 
mapping of systems may well be quite different when thinking about an algae farm system than 
when working with a robotic system. 

The focus group discussed the potential of concept maps being used to demonstrate the learner’s 
progression of understanding. The learning progression was articulated from a definitional 
understanding of a concept (the what), to being able to provide a richer explanation or rationale 
of the concept with examples (the why), to being able to articulate how concepts are applied in 
various situations (the how). Closely related to the demonstration of progression, the focus group 
pointed out the need for the learner to provide a verbal explanation of the maps because, alone 
the map may not be self-explanatory. In the absence of some kind of understanding of the 
thought processes and decision rules that were involved in the map development process, it is 
likely that much of the value of the learning and mapping process will be missed. 

The demonstration of a learner’s conceptual understanding via a concept map led to the 
discussion of identifying learners’ misconceptions. Concept mapping may be of value in helping 
engineering educators understand any firm conceptual “rules” associated with K-12 engineering, 
which, in turn, could be of value in identifying misconceptions. For example, the notion that 
“all” constraints can be optimized is a misconception as is the notion that all constraints can be 
of top priority. Another example is the notion that there is only one correct design solution for 
any given situation. 

The focus group discussed a few specific issues associated with the implementation of concept 
mapping. One of the important issues is determining what concept can or should be mapped and 
which concepts are central. Some of the engineering concepts, such as design or systems, are of 
large enough scale, robust, and are arguably central to engineering to warrant the development of 
maps to indicate a learners’ understanding of these core concepts. Other concepts, such as 
constraints or analysis, are less expansive in scope and are probably best depicted as part of a 
larger concept mapping structure. This is important since one approach to the process is to name 
a central concept at the outset, requiring that the learners maintain the centrality of the concept in 
their maps. Another approach is to issue a set of concepts initially, with the learners making 
decisions about which concepts should be viewed as central and depicting their 
interrelationships. Related to this issue is determining what instructions are provided to the 
learner, such as the type of map, how it is to be created, and the level of detail required. 

Assess Learning? 

The focus group concentrated less directly on the use of concept mapping for assessment 
purposes. However, one important insight that emerged from the discussion was that a primary 
value of concept mapping may be to facilitate learning with assessment of learning being 
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secondary. This said, concept maps appear to be particularly useful in formative assessment, and 
perhaps even in summative assessment, if used in conjunction with other tools. The assumption 
is that significant learning about concepts will occur through the learning process based on 
instructional interventions. Given this, it seems reasonable to think that the concept maps would 
reflect that change in learning and then be assessed as such. Rather than using an approach where 
the learner’s concept maps are compared to an ideal or expert map, assessment should rather 
focus on the observed changes that occurred pre and post to instruction. The focus group did 
indicate that if concept mapping was used for assessment purposes, the instructions and scoring 
process needs to be made explicitly clear to the learner. 

Recommendations and Implications for Research and Practice 

The findings from this focus group study indicate the strong potential for the use of concept 
mapping to help facilitate learning, as well as provide evidence of understanding needed to track 
a learner’s progression in understanding engineering concepts at the pre-college level. The 
identification of engineering concepts and the exploration of using concept mapping to assist in 
learning and assessment raise important recommendations and implications for research and 
practice. As has become clear through this work, a simplistic and disconnected list of concepts is, 
by itself, of little value and fails to grasp the depth and complexity of the ideas. To be 
meaningful, engineering concepts must be (a) situated within their larger social and cultural 
contexts and (b) viewed as interconnected elements in a larger conceptual structure. 

In most cases, the conceptualization process appears to include an aspect of context, since most 
learners think of examples to help make conceptual linkages, decisions, and definitions. Context- 
free abstract conceptualization is not helpful; thus, curriculum development and teacher 
professional development efforts should focus on authentic and meaningful contexts within 
which to embed concepts for instructional purposes. A recent Delphi study conducted by 
Rossouw, Hacker, and de Vries identified nine contexts that included shelter (‘construction’), 
artefacts for practical purposes (‘production’/ ‘manufacturing’), mobility (‘transportation’), 
communication, health (‘biomedical technologies’), food, water, energy, and safety. In terms of 
concept mapping, it seems important to provide the learner with a specific application to use as a 
context for the mapping or conceptualization process. It appears reasonable then that a given 
concept map will be a function of a specific context or project. 

Given the contextual nature of engineering concepts and that there are often numerous legitimate 
ways of conceptualizing and representing the interrelationships among concepts, the educational 
applications of concept mapping need further exploration. For example, the validity of concept 
maps depends on the extent to which the definitions of concepts are clearly understood. While 
the meanings of engineering concepts can be developed generally, a one exact definitional 
understanding of the concept is not perhaps appropriate given that concepts are constructs (and 
therefore functions of academic and social communities). For assessment purposes, this is 
particularly challenging. One approach to evaluating concept maps could be for an expert or 
panel of experts to develop and validate a “correct” map that could be subsequently used as a 
standard by which other maps could be evaluated against. Another approach could be the 
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development of a sequence of concept maps over a span of time to track changes in a learner’s 
understanding of the concept, from definitional terms to its connections to other concepts. 

As precollege engineering education progresses in K-12 educational settings, it is important that 
the conceptual core of engineering be central to its further implementation. The question of how 
concepts drive curriculum, instruction, and assessment needs further work. Appropriately 
answering this question is certainly a large undertaking. Additional work is needed to further 
define those engineering concepts that are core to a robust understanding of engineering and the 
appropriate contexts within which meaningful and age appropriate learning can be developed. 

An effort similar to the AAAS’s Atlas of Science Literacy would greatly assist this effort. This 
groundwork is important for future curriculum development, pre-service teacher preparation, and 
teacher professional development efforts so that they operate from a common base of conceptual 
development. 
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